In 2013, public cloud services have generated over $40B in revenue, which is expected to grow, according to market estimates, to $107B per year by 2017. Just the three largest cloud service vendors: Salesforce, Amazon Web Services, and Akamai derive over $6B in annual revenue from their service offerings. By the year 2017, cloud data analytics will make, by some estimates, dozens of billions dollars in revenue. Access to extensive analytic reports on every aspect of a cloud service functioning and its interaction with customers, other services, software and hardware are useful for the success of such services. Different types of statistics and associated data are distributed between engineering, quality assurance, operations, marketing, business teams and other departments across a service vendor organization. Additionally, business and channel partners, investors, auditors, enterprise and individual customers and, increasingly, a general public are gaining access to relevant portions of service analytics for the mutual benefits of all parties.
Broadening availability of cloud service analytics to various audiences with different positions with respect to the vendor business, diverse degrees of access to the data and dissimilar credentials are stimulating data security considerations with respect to collection, processing and presenting of analytic data and reports. In particular, a direct exposure to the analytics system and its data storage of highly sensitive and secure data stored in principal databases and file repositories of a cloud service may bring an increased risk of massive data breaches. Recent high-profile intrusions suffered by Twitter, Facebook, Apple and other key Internet players have demonstrated new malicious strategies and increased sophistication of online attacks, and growing engineering and IT potential of attackers, which cannot be ignored by architects and developers of analytics systems.
Due to diversified access to analytics data and reports, collecting real-time information from a cloud service by querying main databases and file repositories of the service for inline data analytics may be highly undesirable from the security standpoint. There may be additional channels for penetrating the service with harmful code that may eventually expose all or significant amounts of sensitive customer data stored with the service. Key reasons for such increased risks may include an overall lengthening of the security perimeter, which, in case of direct inline queries, has to include both the main data servers of the cloud service and the analytics system; and a growing access to diverse layers of the analytics data by various audiences both inside and outside of a service vendor company.
Additionally, direct polling of a cloud service for analytics purposes may not satisfy some of the needs of an advanced analytics system for many important categories of cloud services. These services include popular cross-platform personal and business centric content management systems and other productivity cloud services, such as the Evernote Service developed by Evernote Corporation of Redwood City, Calif. In addition, with many millions of users accessing a popular cloud service daily from dozens of platforms, such as Macintosh and Windows desktops, iPhones and iPads, Android, Windows, Blackberry, and other phones, Web browsers, etc., changes in service's databases may occur many times each millisecond and not all of the changes may be visible to direct queries. For example, data deletions, after emptying system trash bins or logging out of the service, may disappear from the system, which keeps the new data and may purge the old data. Other transactions, such as requests for advanced image recognition, represent deferred system activities that can be consistently measured over a certain period of time and may escape direct querying of a service's databases. Therefore, special dynamic mechanisms and logic may be needed to keep track of all system events and customer interactions with a cloud service in order to deliver useful information to the analytics system.
Accordingly, it is desirable to develop secure and efficient methods and processes for building data sets and access procedures for analytics systems associated with cloud services.